1. Field of the Invention
The present invention relates to a pseudo 3D image creation device, a pseudo 3D image creation method, and a pseudo 3D image display system. More particularly, the present invention relates to a pseudo 3D image creation device and a pseudo 3D image creation method for creating a pseudo 3D image from a regular still image or moving image, that is, an image (a non-3D image) having depth information supplied neither explicitly nor, unlike a stereo image, implicitly, and to a pseudo 3D image display system that displays a created pseudo 3D image.
2. Description of the Related Art
To allow a non-3D image to be viewed as a 3D image in a 3D display system, a pseudo 3D image is created from a regular still image or moving image, that is, an image (a non-3D image) having depth information supplied neither explicitly nor, unlike a stereo image, implicitly. In addition to making an image to be viewed as a 3D image, many approaches are now under research and investigation to estimate the three-dimensional structure of a scene from a two dimensional image for composing an image or carrying out virtual viewpoint movement (for example, see Y. Horry, K. Anjyo, K. Arai: “Tour Into the Picture: Using a Spidery Mesh Interface to Make Animation from a Single Image”, SIGGRAPH' 97 Proceedings, pp. 225–232 (1997)). The method called “Tour Into the Picture” described in this document allows the user to remove foreground objects from a photographed image, to specify a vanishing point in the perspective and, based on the specified vanishing point, to estimate the general configuration of a scene for carrying out viewpoint movement.
In contrast to the “Tour Into the Picture” described above in which the depth structure has a tube-like shape whose cross section is a rectangle, a pseudo 3D image creation device and a pseudo 3D image creation method using a perspective-based approach, in which the depth structure has a tube shape whose cross section is a border line according to the depth, are also conventionally known (for example, see Japanese Patent Laid-Open Publication No. Hei9(1997)-185712). According to the invention described in Japanese Patent Laid-Open Publication No. Hei9(1997)-185712, border line distance information is added to mesh image data to produce three-dimensional polygon 3D data and, by applying color image data, obtained from photograph images, to this three-dimensional polygon 3D data, the three-dimensional polygon 3D object composed of the three-dimensional polygon 3D data is rendered such that the color image data is pasted on the inside of the three-dimensional polygon 3D object, thereby to produce three-dimensional image data.
As a classical way to convert a two-dimensional image to a three-dimensional image, the so-called “shape from motion” is known (for example, see C. Tomasi and T. Kanada: “Shape and Motion from Image Streams under Orthography: A Factorization Method”, Int. Journal of Computer Vision. Vo). As its name implies, this method estimates a depth from motion information and configures a 3D image using the motion information on a moving image. However, because it is difficult to automatically estimate a reliable depth without editing and only from the motion, a pseudo 3D image creation device and a pseudo 3D image creation method for making this editing easy are also known (for example, see Japanese Patent Laid-Open Publication No. Hei7(1997)-222201). Japanese Patent Laid-Open Publication No. Hei7(1997)-222201 discloses a conversion method for converting a two-dimensional image to a three-dimensional image. According to this method, an original two-dimensional image is divided into a plurality of scenes and then, depending upon whether each scene is to be used directly as a two-dimensional image or to be converted to a three-dimensional image, the conversion ratio of a three-dimensional image, which will be generated from the two-dimensional image, to the two-dimensional image is adjusted.
However, there are problems with the conventional pseudo 3D image creation devices and the pseudo 3D image creation methods described above in that it is difficult to automatically determine a vanishing point for various images, in that perspective structure estimation is not always suitable for all received scenes, and in that, even if perspective structure estimation is suitable, it is difficult to automatically compose a correct depth structure model for making the image to be viewed naturally as a 3D object.
Another problem with the method (“shape from motion”) described in C. Tomasi and T. Kanada: “Shape and Motion from Image Streams under Orthography: A Factorization Method”, Int. Journal of Computer Vision. Vo is that it is difficult to automatically estimate the depth reliably as described above and, fundamentally, creating a 3D object is impossible for a still image or a relatively-stopped part. In addition, it is difficult to perform processing, which involves motion estimation, at a high speed and this problem tends to prevent an image from being processed in real time.